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Abstract 

The  National  Institute  of  Standards  and  Technology  (NIST)  performs  calibrations 
that  achieve  the  highest  level  of  measurement  quality  for  the  makers  and  users  of 
precision  devices.  The  information  system  that  administers  these  calibrations  should  be  of 
the  same  high  quality  and  reliability.  The  devices  that  NIST  calibrates  cover  a wide 
spectrum,  ranging  from  physical  measurement  standards  to  software  algorithms.  Such  a 
diverse  set  of  equipment  types  put  a tremendous  strain  on  the  development  of  any 
supporting  software,  having  to  cover  a wide  variety  of  extremes.  This  paper  describes  the 
information  system  used  by  NIST  to  support  their  calibration  program.  It  describes  the 
processes  used  to  design  and  implement  the  system  along  with  details  of  its  use  by  the 
technical  and  administrative  staff.  Although  the  system  was  initially  designed  for  a single 
calibration  program,  it  has  been  readily  extended  for  use  throughout  the  entire  NIST 
calibration  program.  The  system  is  World  Wide  Web  (WWW)  based  using  an  Standard 
Query  Language  (SQL)  database  and  the  Practical  Extraction  and  Reporting  Language 
(PERL),  and  therefore  not  reliant  on  any  particular  commercial  product. 


Background 

NIST  has  performed  calibrations  of  precision  instrumentation  and  standards  for 
many  years.  The  system  maintained  by  the  Calibration  Program,  the  central  office  for 
administrative  support  of  calibrations,  was  a System  2K  (S2K)  database,  and 
administratively  kept  track  of  items  sent  in  for  calibration.  Since  it  was  designed  over 
fifteen  years  ago  and  had  surpassed  its  intended  useful  life,  a new  system  was  needed. 


3rd  November,  1999 


' 


2 


An  Information  System  to  Support  Calibrations 


There  are  fourteen  different  divisions  at  NIST  that  perform  calibrations,  with  each 
division  averaging  two  or  three  separate  calibration  groups.  Each  division  informally 
maintains  their  own  financial  and  historical  calibration  data  which  is  vital  to  the  fiscal  and 
technical  management  of  a division.  Each  group  implemented  their  own  systems,  with  no 
regard  for  consistency  or  interoperability.  This  led  to  a wide  variety  of  software  used  by 
each  division  to  track  and  maintain  their  calibrations,  along  with  the  Calibration 
Program’s  System  2K  (S2K)  database.  The  Calibration  Program’s  database  focused 
primarily  on  the  needs  of  NIST  as  a whole  and  not  on  the  individual  needs  of  the 
divisions.  An  updated  system  was  needed  to  meet  the  technical  requirements  of  the 
individual  calibration  programs. 

One  of  the  divisions,  the  Electricity  Division,  performs  approximately  20%  of 
calibrations  at  NIST.  In  May  of  1995,  this  division  formed  a committee  to  explore  ways 
to  improve  the  quality  of  the  calibration  process  within  the  Electricity  Division.  This  team 
was  part  of  a Total  Quality  Management  (TQM)  effort  within  the  Electronics  and 
Electrical  Engineering  Laboratory  (EEEL).  A Calibration  Process  Improvement  Team 
(CPIT)  was  formed  and  comprised  of  a facilitator,  and  representatives  from  Management, 
the  Calibration  Program,  and  from  each  of  the  calibration  areas  within  the  division.  The 
calibration  areas  of  the  division  include  Resistance,  Impedance,  Voltage,  Precision  Ratio, 
Phase  Meters  and  Power  and  Energy.  The  CPIT  met  frequently  over  the  next  year.  The 
goal  of  the  CPIT  was  to  identify  problems  with  the  calibration  process  and  recommend 
solutions  to  rectify  the  identified  problems.  This  effort  served  to  strengthen  the  calibration 
process,  making  it  acceptable  to  the  division  staff,  the  Calibration  Program  personnel,  and 
the  calibration  customer. 


Identifying  the  Problems 

The  first  problem  identified  by  the  CPIT  was  the  time  the  technical  calibration 
personnel  were  spending  on  administrative  requirements  of  calibrations.  Too  much  time 
was  being  expended  on  administrative  functions  rather  than  on  the  calibration  itself. 
Responsibilities  of  the  technical  personnel  included  submitting  cost  information  and  a 
calibration  report  for  each  item  calibrated.  Most  costs  are  fixed  (i.e.,  have  a set  price),  but 
many  are  "at  cost"  and  the  final  price  must  be  computed  after  the  calibration  is  done.  The 
calibration  personnel  determined  the  cost  of  the  calibration  by  computing  a "time  and 


3rd  November,  1999 


- 


Brady 


3 


materials"  cost  using  a special  form,  which  accounted  for  all  overhead  costs  incurred  by 
NIST.  This  form  could  take  a considerable  amount  of  time  to  fill  out,  with  much  of  the 
time  spent  tracking  down  the  overhead  rates.  The  submitted  forms  had  to  be  hand  typed, 
sent  to  the  Calibration  Program,  and  then  on  to  Accounts  Receivable  for  billing. 

Generating  the  calibration  report  also  presented  problems.  To  prepare  a calibration 
report  the  technical  staff  used  a plethora  of  text  preparation  packages.  These  packages 
ranged  from  simple  FORTRAN  programs  to  complex  word  processors.  And  once  the 
report  was  generated,  the  technician  archived  a copy  on  their  local  system  for  future 
reference.  Each  report  was  either  generated  from  a template  of  a previous  report  or 
generated  from  scratch.  As  technical  people  left,  their  replacements  used  the  existing 
system  to  generate  reports  or  developed  their  own  style.  The  reports  themselves  carried 
no  specified  format.  Each  calibration  service  maintained  its  own  style,  with  no 
consistency  across  groups,  much  less  across  the  division.  Some  services  were  inconsistent 
from  calibration  to  calibration. 

The  CPIT  felt  that  this  process  could  be  automated,  relieving  the  calibration  staff 
from  some  of  the  administrative  duties.  In  the  process,  the  calibration  reports  could  be 
"standardized"  so  that  reports  from  the  Electricity  Division  would  maintain  a common 
look  and  reporting  style.  Most  reports  contained  information  explaining  the  measurement 
process  used,  the  uncertainties,  temperature  and  other  ambient  conditions.  The  reports 
were  modified  to  include  a "fact  sheet"  to  explain  the  measurements  and  other  factors, 
resulting  in  clear,  concise  reports  for  the  calibration  customer. 

Historical  data  on  the  equipment  being  calibrated  was  stored  with  previous 
calibrations,  usually  on  three  by  five  index  cards  in  a filing  cabinet.  This  meant  no  readily 
available  means  to  regenerate  calibration  reports.  Also,  historical  information  on  a item 
sent  in  for  calibration  was  not  readily  accessible.  Using  a database,  historical  data  could 
be  maintained  and  easily  retrieved.  Calibration  personnel  could  have  quick  and  easy 
access  to  prior  results  for  use  in  the  current  calibration.  Backups  would  prevent  data  loss 
or  inadvertent  destruction.  Historical  data  would  not  be  affected  by  changes  in  personnel. 

The  last  area  identified,  and  the  most  major,  was  in  customer  service.  The  calibration 
process  can  be  viewed  as  a business,  and  as  in  any  business,  one  of  the  major  goals  is 
customer  satisfaction.  Under  the  old  system,  customers  knew  little,  if  anything,  about 
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their  instrument  sent  in  for  calibration  until  it  was  returned.  The  CPIT  felt  that  increased 
communication  with  the  customers  was  vital  for  the  calibration  services.  A technical 
contact  at  each  company  could  be  identified  and  information  on  the  calibration  sent  to 
them  directly.  Communications  with  the  customer  was  "a  must"  in  order  to  better  serve 
them. 

Some  other  areas  noted  by  the  CPIT  were:  1)  reduce  effort  needed  for 
administration,  2)  store  customer  addresses  and  contact  information  in  the  database,  3) 
improve  workflow  tracking,  4)  notify  staff  of  new  work  via  email,  and  5)  fiscal  and 
workload  reports. 


Design  Goals 

As  a result  of  the  CPIT  meetings,  it  was  recommended  that  a database  be  developed 
to  support  the  entire  calibration  process.  Such  a database  would  have  to  use  industry 
standards,  be  easy  to  use,  provide  access  to  all  types  of  computers  and  operating  systems 
(e.g.,  not  be  system  dependent),  use  email  for  notifications,  provide  protection  to 
calibration  data  and  print  to  any  networked  printer.  A simple  task  by  any  means! 

Another  recommendation  from  the  CPIT  was  to  focus  on  improving  customer 
service.  The  calibration  system  would  have  to  provide  status  information  to  the  customer 
on  their  equipment  in  for  calibration. 

With  very  little  internal  funding  available,  the  design  process  did  have  the  added 
constraint  of  "little  to  no  " software  costs.  With  as  many  as  60  users  in  one  division  alone, 
any  added  "per  user"  license  costs  would  make  the  software  goal  unobtainable. 

The  first  goal  of  zero  costs,  but  great  software,  was  achieved  by  using  the  database 
provided  by  the  NIST  Information  Technology  Laboratory  (ITL).  Using  overhead 
funding,  ITL  had  purchased  an  SQL  database  for  use  by  the  laboratories.  With  the 
database  in  place,  we  needed  to  provide  access  via  the  network  to  all  types  of  computers, 
running  differing  operating  systems.  To  avoid  having  to  port  any  client  software  to  each 
harware  and  software  platform,  we  chose  a WWW  browser  as  the  interface  to  the 
database.  Browsers  were  available  for  every  machine,  and  we  would  not  expend  any 
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effort  porting  the  application.  Browsers  are  available  to  all  NIST  personnel,  and  only 
older  versions  of  the  browsers  would  present  a problem  (e.g.,  we  took  full  advantage  of 
new  capabilities  in  newly  released  versions).  Personnel  were  advised  to  upgrade  to  the 
latest  versions  of  their  browser  for  best  results. 

The  WWW  interface  saved  us  numerous  hours  of  development,  and  allowed  us  to 
build  our  software  as  new  developments  to  the  WWW  were  being  made  in  parallel.  The 
Common  Gateway  Interface  (CGI)  provided  the  ability  to  access  the  database  and 
dynamically  generate  the  resulting  page  using  HTML.  The  WWW  was  a new  and 
exciting  technology  that  fit  the  requirements  of  our  database  perfectly.  The  database  also 
put  a new  spark  into  the  mundane  part  of  the  calibration  process. 

System  Implementation 

The  system  has  been  in  development  for  three  years  now,  mostly  part-time  by  two 
people,  programming  as  time  allowed.  We  have  expanded  it  vastly,  taking  advantage  of 
developments  in  HTML  and  WWW  technologies  as  they  occur  (e.g.,  tables  did  not  exist 
when  we  started,  but  were  incorporated  after  a few  months  of  development).  The  most 
time  consuming  task  is  the  CGI  programming,  which  is  done  in  PERL.  Each  page  is 
programmed  individually,  but  common  routines  are  in  a library  and  accessed  by  many 
pages.  We  have  divided  the  system  into  four  major  areas  and  can  allow  access 
independently  to  each.  The  administrative  area  is  for  managers  and  Administrative 
Officers,  and  provides  numerous  revenue  reporting  facilities.  The  data  entry  area  is  for  the 
maintainers  of  the  database.  All  purchase  order  information  is  entered  and  changed  in 
this  section.  Access  to  this  area  is  restricted  to  the  few  who  perform  this  task,  making  the 
data  more  reliable  and  limiting  unwanted  changes  or  mistakes  in  the  database.  The  third 
section  is  for  the  Technical  Personnel,  who  can  check  on  waiting  calibrations,  enter 
calibration  data  and  generate  calibration  reports.  The  last  area  is  for  Customers,  who  can 
access  a status  page  for  each  current  calibration.  This  area  will  be  discussed  in  a later 
section. 

Dividing  the  database  into  discrete  sections  allows  us  to  maintain  proper  security 
controls  on  the  data  contained  within  it.  Each  person  has  a specific  function  in  the 
calibration  process,  be  it  entering  purchase  order  data,  or  submitting  calibration  data,  or 
running  fiscal  reports.  Using  the  security  mechanisms,  we  can  allow  access  by  personnel 
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only  to  the  areas  they  need  to  perform  their  function,  (e.g.,  administrative  staff  cannot 
enter  technical  data  or  print  calibration  reports,  and  technical  staff  cannot  generate 
administrative  paper  work,  other  than  the  NIST  Form  77  for  time  and  materials  costs  they 
require.)  This  separation  of  powers  is  a fundamental  principle  of  a database,  and  ensures 
proper  controls  and  data  integrity. 

The  last  two  years  have  been  spent  generating  pages  for  the  technical  and 
administrative  staff  to  perform  their  specified  functions.  The  pages  are  designed  and 
implemented,  used  by  the  staff,  feedback  provided  to  the  programmers,  refinements  made 
and  the  pages  changed,  then  the  process  started  again.  Using  this  process  we  have 
developed  an  information  system  designed  specifically  for  the  NIST  staff,  providing  both 
technical  and  administrative  support  in  a single  database.  Each  page  has  been  finely  tuned 
to  the  technical  and  ergonomic  needs  of  the  staff,  a feat  not  possible  with  a contract 
programming  exercise. 


Security 

As  we  mentioned  above,  the  calibration  process  is  a business,  and  we  could  not 
allow  any  one  customer  to  see  another  customer’s  calibration  data.  In  fact,  one  customer 
is  not  permitted  to  know  another  customer  had  an  item  in  for  calibration.  So,  limiting 
access  to  the  database  to  unauthorized  users  was  a top  priority. 

The  database  was  an  "Intranet"  resource,  so  only  NIST  personnel  on  the  campus 
needed  access.  The  first  security  step  was  NIST’s  installed  firewall  around  the  campus 
restricting  access  to  the  database  from  the  Internet.  The  second  security  measure,  each 
instance  of  the  database  can  be  further  restricted  by  limiting  access  to  the  database  by 
subnet.  This  prevents  users  in  one  lab  from  gaining  access  to  another  lab’s  version  of  the 
database.  The  last  tier  of  the  security  triad  is  the  username  and  password.  Each  user  is 
issued  a username  and  password  to  gain  access  to  the  database.  These  three  security 
measures,  firewall,  domain  restriction,  and  usemame/password  provide  suitable  security 
for  the  database.  The  usemame/password  is  implemented  via  the  web  server  security 
mechanism  and  is  undergoing  constant  development  and  upgrades,  as  is  the  firewall.  The 
last  problem  was  to  provide  internet  access  for  customers  to  calibration  status 
information,  while  maintaining  the  high  level  of  security. 
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Since  the  database  sits  behind  an  internet  firewall,  providing  access  to  customers 
was  not  as  easy  as  just  generating  a web  page  for  them  to  view. 

Figure  1.  External  Customer  Access 


NIST  Internal  Server 


ISSC 

Database 


Internet  firewalls  require  a pass  key  for  access  and  are  quite  complicated.  Since  the 
customers  could  not  access  our  internal  pages,  we  had  to  make  the  pages  accessible 
outside  the  firewall.  Every  hour,  on  the  internal  server  behind  the  firewall,  a job  is  run  that 
generates  a web  page  for  each  open  calibration  and  for  each  calibration  closed  within  the 
last  thirty  days.  This  allows  customers  access  to  shipping  information  while  the  item  is  in 
transit.  The  pages  are  then  propagated  to  the  external  server.  The  web  page  details  the 
status  of  the  calibration,  whether  it  is  awaiting  calibration,  in  progress,  or  in  the  review 
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process.  Relevant  dates  are  also  available,  including  among  others  the  date  the  report  was 
printed,  the  date  the  calibration  was  finished  and  the  date  the  equipment  was  shipped 
back  to  the  customer.  NIST  technical  personnel  responsible  for  the  calibration  can  be 
reached  via  the  provided  phone  number  or  email  address.  The  latest  shipping  and  billing 
addresses  are  also  available  for  customer  verification.  In  order  to  ensure  that  a customer 
has  access  to  only  their  web  page,  a username  and  password  is  required.  When  an  item  is 
received  for  calibration,  a NIST  form  64  acceptance  letter  is  mailed  to  the  customer.  This 
form  contains  the  estimated  cost  of  the  calibration,  verifies  receipt  of  the  item  for 
calibration,  and  provides  the  NIST  technical  contact(s)  responsible  for  the  calibration.  A 
machine  generated  password,  along  with  a username  is  provided  on  this  form.  After 
receipt  of  the  NIST  64  acceptance  letter,  the  customer  can  then  access  the  Universal 
Resource  Locator  (URL)  of  their  status  page,  24  hours  a day,  seven  days  a week.  This 
web  page  is  accessible  to  the  company  who  submitted  the  respective  equipment  for  thirty 
days  after  the  calibration  is  completed. 


Expansion  to  NIST  Calibration  Program 

Once  the  ISSC  was  in  use  by  our  division,  other  calibration  divisions  at  NIST  heard 
of  its  existence  and  requested  demonstrations  of  the  system.  After  a few  demonstrations, 
one  other  division  wanted  to  know  if  there  was  any  way  possible  for  them  to  use  it.  The 
administrative  requirements  for  each  division  across  NIST  are  identical,  with  the  only 
differences  in  the  items  being  calibrated.  Since  the  only  differences  were  in  the 
information  stored  in  the  database,  replicating  the  system  involved  populating  the 
database  with  their  initial  cost  information,  item  descriptions,  and  staff  listings.  Having 
separate  instances  for  each  division  was  just  a distributed  database  implementation,  with 
none  of  the  data  replicated  among  databases.  Each  division  can  run  fiscal  and  turn- 
around time  reports,  that  only  gather  data  from  their  calibrations,  reporting  on  only  their 
data.  We  also  had  to  build  new  templates  for  calibration  reports  that  differed  from  our 
own.  For  example,  thermodynamic  calibration  reports  were  very  similar  to  ours,  only 
different  wording  in  the  report.  This  let  us  start  a few  services  in  other  divisions  printing 
calibration  reports  immediately.  Again,  the  reporting  style  and  format  of  the  reports 
would  spread  across  NIST. 

The  Calibration  Program  requested  an  interface  to  the  Information  System  to 
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Support  Calibrations  (ISSC).  The  ISSC  contained  all  the  administrative  information  they 
needed,  so  their  S2K  database  would  be  discontinued,  and  the  ISSC  would  become  the 
NIST  standard.  Since  each  division  was  generating  reports,  we  needed  to  create  "Views" 
for  the  calibration  program  that  looked  at  all  the  divisions  as  a single  entity.  This  would 
allow  the  calibration  program  to  generate  reports  combining  all  the  divisions,  and  report 
on  the  NIST  calibration  program  as  a whole.  Each  division  would  be  autonomous,  control 
their  own  data,  and  the  calibration  program  would  oversee  the  fiscal  reporting  for  NIST. 

The  last  area  that  needed  to  be  addressed  was  the  Accounts  Receivable  and  billing 
functions.  The  old  S2K  database  fed  data  weekly  to  the  main  NIST  accounts  receivable 
database  to  enable  customer  billing.  The  Chief  Financial  Officer’s  (CFO)  personnel  are 
the  only  ones  that  have  access  to  the  billing  functionality.  We  designed  a WWW  based 
front-end  for  them,  which  allows  them  to  bill  folders  as  soon  as  the  technical  divisions 
close  them  out.  We  improved  the  quality  of  the  bills  by  generating  Portable  Document 
Format  (PDF)  versions  of  the  bills,  based  on  customer  type.  The  correct  bill  is  generated, 
and  a PDF  version  sent  back  to  the  browser  for  printing  and  mailing. 


Committee  Support  - the  ISSCOC 


Once  the  system  was  in  use  by  other  divisions,  a committee  was  formed  to  oversee 
operations  and  upgrades  to  the  system  - The  Information  System  to  Support  Calibrations 
Oversight  Committee  (ISSCOC).  One  member  from  each  calibration  division,  the  CFO’s 
office,  the  Calibration  Program  and  ITL  comprised  the  committee.  The  committee 
prevented  any  influence  from  any  single  division,  and  made  the  database  truly  a NIST 
resource.  The  committee  meets  monthly,  and  new  issues  are  discussed  and  voted  upon. 
Changes  and  bug  fixes  are  discussed,  and  only  approved  on  majority  approval.  The 
committee  also  discusses  items  related  to  calibrations,  and  not  necessarily  to  do  with  the 
ISSC.  It  has  developed  into  an  excellent  forum  to  discuss  many  calibration  issues. 
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Future  Goals 

The  main  thrust  of  the  next  year  will  be  creating  templates  for  calibration  reports. 
Data  already  exists  in  the  database,  generating  calibration  reports  will  be  an  ongoing 
programming  project.  We  start  with  divisions  that  have  reports  similar  to  ones  already 
being  generated,  and  will  then  move  to  the  more  difficult  reports.  The  goal  is  to  have 
everyone  wishing  to  print  calibration  reports  doing  so  by  the  end  of  the  next  fiscal  year. 

Another  area  of  research  uses  iButton  technology.  An  iButton  resembles  a watch 
battery,  with  a physical  size  about  that  of  a nickel.  The  iButton  contains  a small  JAVA 
virtual  machine,  and  about  6K  of  available  non-volatile  memory.  Information,  such  as  a 
calibration  report  or  calibration  data  can  be  written  to  the  iButton,  and  physically  stored 
with  the  artifact.  Access  to  the  iButton  is  protected  by  a username  and  password 
controlled  by  the  JAVA  virtual  machine.  Data  required  by  the  ISSC  such  as  model 
number,  serial  number,  and  manufacturer  can  be  stored  on  the  iButton  and  read 
electronically  when  the  item  is  received  for  calibration.  Customers  could  also  be  provided 
readers  and  have  access  to  all  information  stored  on  the  iButton.  The  costs  of  the  iButtons 
is  minimal,  and  the  readers  are  inexpensive  as  well. 


Summary 

The  ISSC  will  be  in  use  across  NIST  by  the  start  of  fiscal  year  2000.  All  calibrations 
done  at  NIST  will  be  entered,  tracked  and  billed  using  the  ISSC.  The  customer  access 
pages  will  be  available  by  August  1999,  as  soon  as  the  NIST  firewall  is  complete. 

We  did  achieve  the  goal  of  "minimal  software  costs",  using  shareware  (PERL),  and 
the  ITL  ORACLE  database.  The  only  cost  was  the  staffing  to  initially  design  and  build 
the  ISSC.  With  no  recurring  software  costs,  future  funding  can  be  spent  on  upgrades  to 
new  web  technology,  greatly  extending  the  useful  life  of  the  ISSC.  Some  programming 
tasks  can  be  assigned  to  summer  students,  who  have  proven  to  be  an  excellent  source  of 
programming  expertise. 

To  access  the  ISSC,  a staff  member  needs  a "vanilla"  browser.  All  documents 
returned  to  the  browser  are  in  the  PDF.  After  viewing  the  document,  the  user  can  print  a 
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hardcopy  version  to  any  available  printer. 

The  complexity  of  the  ISSC  prevents  it  from  being  purchased  as  "off-the-shelf 
software.  Contracting  out  this  work  would  have  been  relatively  expensive.  The  in-house 
development  of  the  ISSC  brought  together  the  many  people  involved  in  the  calibration 
process  to  work  together  towards  a common  goal.  The  end  product  is  a high-quality 
calibration  program,  supported  by  the  NIST  community. 
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